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The  mission  of  the  Office  of  the  Controller  (Controller's  Office)  of  the  City  and  County  of  San  Francisco 
is  to  ensure  the  City's  financial  integrity  and  promote  efficient,  effective,  and  accountable  government. 
The  Controller's  Office's  vision  is  to  be  a  model  for  good  government  and  to  make  the  City  a  better 
place. 

The  City  Services  Auditor  (CSA)  was  created  within  the  Controller's  Office  through  a  2003  Charter 
amendment  that  envisions  a  broad  effort  to  measure,  audit,  and  report  on  San  Francisco  government's 
performance.  The  mandate  is  to  analyze  the  City's  public  service  delivery,  compare  and  benchmark 
San  Francisco  to  best  practices  nationwide,  provide  information  to  citizens  in  new  ways  and  help  drive 
improvements  in  City  government.  This  is  among  the  most  ambitious  voter-approved  efforts  of  its  kind 
in  local  government. 

CSA  has  broad  authority  for: 

•  Reporting  on  the  level  and  effectiveness  of  San  Francisco's  public  services  and  benchmarking 
the  City  to  other  public  agencies  and  jurisdictions. 

•  Conducting  financial  and  performance  audits  of  City  departments,  contractors,  and  functions  to 
assess  efficiency  and  effectiveness  of  processes  and  services. 

•  Operating  a  whistleblower  hotline  and  website  and  investigating  reports  of  waste,  fraud,  and 
abuse  of  City  resources. 

•  Ensuring  the  financial  integrity  and  improving  the  overall  performance  and  efficiency  of  City 
government. 

CSA  is  funded  through  a  commitment  of  2/10ths  of  one  percent  of  the  City's  annual  budget.  In  fiscal 
year  2007-08,  approximately  $12  million  was  budgeted  under  the  Charter  requirement  and  $8.5  million 
was  spent,  with  the  balance  returned  to  City  funds. 
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ANNUAL  WORK  PLAN,  FISCAL  YEAR  2009-10 


Summary 

The  City  Services  Auditor  (CSA)  developed  its  Annual  Work  Plan  (Work  Plan)  by  considering  audits 
and  other  responsibilities  mandated  by  the  City  Charter  and  municipal  codes,  the  results  of  a  limited 
risk  assessment,  and  input  from  City  leadership  and  stakeholders. 

•  Section  I  provides  background  information  and  identifies  CSA's  highest  priority  initiatives  for 
fiscal  year  2009-10. 

•  Section  II  describes  CSA's  two  main  operational  units,  Audits  and  City  Performance,  and  the 
major  areas  each  unit  has  identified  to  focus  its  work. 

•  Section  III  details  CSA's  funding  for  carrying  out  its  work  during  the  year,  as  well  as  its  staff 
resources  for  each  of  CSA's  operational  units. 

•  Section  IV  lists  the  major  audits  and  projects  planned  for  the  year,  along  with  their  planned 
budgets. 


I.  Background  and  Overall  Fiscal  Year  2009-1 0  Priorities 

CSA  developed  its  Work  Plan  by  considering  audits  and  other  responsibilities  mandated  by  the  City 
Charter  and  municipal  codes,  the  results  of  a  limited  citywide  risk  assessment,  and  input  from  the 
Board  of  Supervisors,  the  Mayor,  and  various  City  department  heads  and  managers.  This  approach 
ensures  adequate  coverage  of  City  processes  and  programs  in  high-risk  areas  and  recognizes 
concerns  raised  by  the  City  leadership. 

The  Work  Plan  may  change  during  the  fiscal  year  as  circumstances  dictate.  Quarterly  reassessment 
of  risk,  requests  from  City  leadership,  changes  within  City  organizations  or  operations,  and  available 
CSA  staff  resources  can  result  in  changes  to  the  plan. 

CSA's  highest  priority  initiatives  for  fiscal  year  2009-10  include  work  to: 

1.  Assist  the  Municipal  Transportation  Agency  in  implementing  citywide  changes  to  routes  and 
services  to  improve  reliability  and  speed  for  its  bus  and  light  rail  systems. 

2.  Implement  recommendations  for  wide-ranging  changes  to  the  organization  of  the  San 
Francisco  Police  Department  based  on  the  consultative  processes  the  department  completed 
last  fiscal  year  and  with  the  direction  of  the  new  Police  Chief. 

3.  Evaluate  service  delivery  and  funding  of  the  City's  complex  $1 .3  billion  public  health  program, 
focusing  on  specific  programs  including  substance  abuse  treatment  and  the  universal  health 
coverage  program,  Healthy  San  Francisco. 

4.  Establish  audit  approaches  that  mitigate  opportunities  for  contractor  overcharges  and 
improve  contractor  oversight  and  compliance  for  the  Public  Utilities  Commission's  10-year, 
multi-billion  dollar  capital  program  and  other  key  City  construction  programs. 

5.  Assist  the  City's  critical  health  and  social  services  sector  through  a  variety  of  support  and 
efficiency  initiatives.  CSA's  work  will  support  City  departments  including  Public  Health, 
Human  Services,  and  Children,  Youth  and  Their  Families;  and  their  web  of  non-profit  service 
providers  who  are  coping  with  reduced  budgets  and  increased  service  needs  in  the  current 
economic  climate. 
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6.    Develop  a  continuous  auditing  program  that  employs  technology  to  help  evaluate,  monitor,  or 
review  an  organization's  activities  on  a  more  frequent  or  ongoing  basis.  Continuous  auditing 
programs  use  automated  tools  to  analyze  large  data  sets  to  promote  the  detection  and 
prevention  of  fraud,  inefficiency,  errors,  and  control  weaknesses.  The  program  will  focus  on 
areas  such  as  payroll,  procurement,  inventories,  and  financial  transactions. 


II.  Audits  and  City  Performance  -  Scopes  of  Work 

CSA  performs  its  work  through  two  primary  operational  units,  the  Audits  Unit  and  the  City 
Performance  Unit.  These  units  are  supported  by  an  Operations  Unit  that  performs  business 
management  and  contract  administration  functions. 

AUDITS  UNIT 

The  Audits  Unit  complies  with  generally  accepted  government  auditing  standards  that  provide  a 
framework  for  conducting  high  quality  audits  with  competence,  integrity,  objectivity,  and 
independence.  The  unit  provides  the  following  audit  services: 

•  Performance  Audits 

Performance  audits  measure  the  extent  to  which  a  program  is  achieving  its  goals  and 
objectives,  the  effectiveness  of  organizations,  programs,  activities,  or  functions,  and  whether 
an  organization  has  complied  with  laws  and  regulations  applicable  to  the  program.  In 
addition,  performance  audits  determine  the  economy  and  efficiency  of  an  organization's 
operations,  assessing  whether  the  organization  is  acquiring,  protecting,  and  using  its 
resources  economically  and  efficiently,  and  the  causes  of  inefficiencies  or  uneconomical 
practices.  The  Audits  Unit  works  with  management  in  developing  recommendations  that  can 
be  used  as  a  basis  for  building  more  efficient  and  effective  operations.  The  general  objective 
of  a  performance  audit  is  to  identify  specific  areas  of  operations  that  can  be  improved. 

•  Financial  Audits 

Financial  audits  assure  the  City  that  departments  properly  account  for  and  report  the  financial 
results  of  their  operations.  As  part  of  that  review,  auditors  evaluate  internal  controls  over 
expenditures  to  assure  that  expenditures  are  properly  authorized  and  controlled.  The  auditors 
also  evaluate  internal  controls  over  revenues  to  assure  revenues  are  maximized  and 
expeditiously  managed.  These  audits  provide  information  about  internal  control,  compliance 
with  laws  and  regulations,  and  adherence  with  contract  and  grant  provisions  relating  to 
financial  transactions,  systems,  and  processes. 


Strategic  Focus  for  Fiscal  Year  2009-10 

The  Audits  Unit  has  identified  the  following  major  areas  in  which  it  will  target  a  large  percentage  of  its 
resources.  Section  IV  of  this  report  lists  the  specific  audits  it  plans  to  conduct  related  to  these  major 
areas.  The  unit  also  plans  to  conduct  additional  audits  that  are  not  listed  in  Section  IV.  While  these 
audits  are  not  of  lesser  importance,  they  require  fewer  hours  to  conduct,  and  would  be  too  numerous 
to  list.  Finally,  the  Audits  Unit  has  reserved  approximately  10  percent  of  its  audit  resources  to  respond 
to  unforeseen  audit  needs  that  may  arise  during  the  year. 

Fraud  Investigations 

•    Operate  a  whistleblower  complaints  hotline  and  website  and  investigate  reports  of  waste, 
fraud,  and  abuse  of  City  resources. 
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Continuous  Auditing  Services 

•  Develop  continuous  auditing  processes  to  automate  regular  testing  so  auditors  can  focus  on 
higher  risk  areas;  determine  the  effectiveness  of  key  controls  by  analyzing  financial 
transactions  continuously  and  independently;  and  reduce  audit  time  and  effort  spent  on 
controls  testing. 

Information  Technology 

•  Assess  the  adequacy  of  internal  controls  for  the  City's  information  technology  systems. 
Procurement  and  Lease  Compliance 

•  Ensure  that  the  City's  leasing  and  lease  administration  activities  are  cost-effective  and  are  in 
accordance  with  contract  provisions  and  City  codes. 

Construction  Audits 

•  Ensure  that  procurement  and  contracting  activities  are  cost-effective  and  conform  to 
applicable  City  codes  and  guidelines. 

Citywide  Risk  Management 

•  Develop  a  robust  Enterprise  Risk  Management  process  to  take  into  account  the  strategic, 
regulatory,  financial,  and  business  risks  to  which  the  City  has  exposure.  This  approach  will 
assist  the  Audits  Unit  to  achieve  potential  maximum  value  for  work  performed. 

Audit  Process 

The  audit  process  typically  consists  of  three  phases:  planning,  field  work,  and  report  writing. 

During  the  planning  phase,  auditors  obtain  background  information  by  reviewing  written  procedures, 
governing  laws  and  regulations,  organization  charts,  and  financial  information  on  the  program  or 
activity  being  audited.  The  auditors  collect,  summarize,  and  evaluate  this  information  to  identify  where 
to  focus  its  audit  efforts  and  develop  an  audit  program  that  includes  audit  objectives  and  the  specific 
procedures  the  auditors  will  carry  out  to  address  those  objectives. 

During  the  field  work  phase,  auditors  identify  criteria  against  which  to  measure  performance  and 
gather  and  analyze  information  (audit  evidence)  to  measure  and  evaluate  performance.  The  results 
are  summarized  in  the  form  of  audit  findings,  which  include  five  elements: 

•  Condition  (what  is) 

•  Criteria  (what  should  be) 

•  Cause  (why  did  it  happen) 

•  Effect  (why  does  it  matter) 

•  Recommendations  (how  to  fix  or  prevent  it  from  happening) 

The  final  phase  is  report  writing.  The  audit  team  prepares  a  draft  audit  report  and  holds  an  exit 
conference  with  management  of  the  audited  entity  to  review  the  findings,  most  of  which  the  audit 
team  would  have  already  discussed  with  management  during  field  work.  The  unit  conducts  an  internal 
quality  control  process  to  verify  that  each  statement  of  fact  in  the  report  is  supported  in  the  audit 
documentation.  The  audit  team  then  provides  a  copy  of  the  draft  report  to  management  to  give  them 
an  opportunity  to  provide  feedback  and  a  written  response  indicating  their  concurrence  or  non- 
concurrence  with  the  recommendations  and  an  implementation  plan.  The  final  audit  report  includes 
management's  response.  Audit  reports  are  public  documents. 
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CITY  PERFORMANCE  UNIT 

The  City  Performance  Unit  provides  technical  assistance  and  analysis  to  City  departments  and 
decision  makers  to  help  improve  the  effectiveness  and  efficiency  of  public  services.  The  unit 
measures  performance  and  produces  information  and  reports  that  serve  the  public's  interest  in 
government  and  in  understanding  the  quality  and  quantity  of  public  services  delivered  by  the  City. 
Projects  focus  on  the  cost,  performance,  and  demand  for  City  services,  programs,  and  facilities,  and 
help  ensure  that  City  resources  are  maximized.  The  unit's  work  includes  analysis,  consultation, 
recommendations,  implementation  support  and  the  development  of  management  tools. 

City  Performance  Strategic  Focus  Areas  for  Fiscal  Year  2009-10 

The  City  Performance  Unit  has  identified  the  following  major  service  areas  for  fiscal  year  2009-10. 
Section  IV  of  this  report  lists  the  specific  projects  related  to  these  major  areas. 

Health  and  Human  Services 

•  Evaluate  the  City's  substance  abuse  treatment  services  to  determine  how  to  best  meet  the 
needs  of  clients  and  improve  results. 

•  Evaluate  methods  for  maintaining  hospital  capacity  and  increasing  access  to  quality  primary 
and  community-based  care 

Nonprofit  Service  Delivery 

•  Improve  nonprofit  service  outcomes  and  contract  management. 

•  Provide  financial  and  organizational  technical  assistance  to  nonprofit  contractors  and 
streamline  fiscal  and  compliance  oversight. 

Parks,  Streets,  and  Sidewalks 

•  Build  the  City's  ability  to  manage  and  improve  parks,  streets,  and  sidewalks  using  established 
maintenance  standards  and  objective  data  on  the  condition  of  parks,  streets,  and  sidewalks. 

Performance  Measurement 

•  Increase  performance-based  budgeting,  contracting,  and  service  delivery  and  improve  the 
City's  reporting  to  the  public  on  performance. 

•  Provide  a  performance  measurement  database  and  advice  to  City  leadership  and  staff 
enabling  them  to  better  manage  resources  using  measures  tracked  in  the  database. 

Public  Safety 

•  Assist  the  San  Francisco  Police  Department  (SFPD)  and  its  stakeholders  in  their  efforts  to 
decrease  crime  and  increase  community  safety.  Implement  strategic  improvements  in  the 
SFPD  as  prescribed  by  expert  studies  and  consultative  processes. 

Public  Transit 

•  Provide  assistance  to  the  Municipal  Transportation  Agency  (MTA)  in  their  efforts  to  increase 
ridership  and  convenience  while  lowering  costs. 

•  Provide  a  multi-year  improvement  plan  and  implementation  support  so  that  MTA  can  better 
deliver  its  services 

City  Performance  Process 

The  City  Performance  Unit  provides  technical  assistance  and  analysis  to  City  departments  and 
decision  makers.  Analysis  projects  address  the  cost,  performance,  and  demand  for  City  services.  The 
approach  to  each  project  is  customized  to  that  project's  requirements.  Projects  follow  accepted  policy 
analysis  methodology,  including  defining  the  problem,  assembling  evidence,  constructing  alternatives, 
and  analyzing  alternatives  against  established  criteria. 
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Through  technical  assistance  projects,  the  unit  provides  consultation,  recommendations, 
implementation  support,  and  the  development  of  management  tools.  The  unit  implements  technical 
assistance  projects  using  accepted  project  management  practices.  Project  phases  include  project 
initiation/definition,  project  planning,  execution,  monitoring  and  controlling,  and  close  out  and 
evaluation.  City  Performance  staff  receive  regular  training  on  project  management  skills  to 
continuously  improve  project  execution. 

III.  Budget  and  Allocation  of  Resources 

CSA's  fiscal  year  2009-10  Work  Plan  includes  audits,  projects,  and  performance  measurement 
assistance  that  address  a  wide  range  of  concerns  consistent  with  CSA's  areas  of  responsibility.  The 
Work  Plan  is  designed  to  address  the  highest  priority  areas,  while  aligning  the  scope  of  work  with 
staffing  and  budget  resources. 

City  Services  Auditor  Budget 


The  table  below  shows  the  funding  for  CSA  in  fiscal  year  2009-10  under  the  2/10ths  of  one  percent 
budget  allocation  mandated  in  the  Charter. 


Department 

FY09-10  Budget 

Percent  of  Total 

Airport 

$817,452 

6.59% 

Children  and  Families  Commission 

47,690 

0.38% 

Children,  Youth  and  Their  and  Families 

258,064 

2.08% 

Child  Support  Services 

29,994 

0.24% 

Building  Inspection 

79,466 

0.64% 

Community  Health 

1,679,915 

13.55% 

San  Francisco  General  Hospital 

798,312 

6.44% 

Laguna  Honda  Hospital 

405,134 

3.27% 

Public  Works 

208,707 

1.68% 

Human  Services 

1,296,837 

10.46% 

Environment 

28,481 

0.23% 

General  Fund  -  Citywide  &  Departments 

3,410,315 

27.51% 

Public  Library 

159,319 

1.29% 

Municipal  Transportation  Agency 

1,307,405 

10.55% 

Port 

157,595 

1.27% 

Parking  and  Traffic 

211,068 

1.70% 

Public  Utilities  Commission 

1,008,123 

8.13% 

Recreation  and  Parks 

407,275 

3.29% 

Retirement  System 

37,462 

0.30% 

Rent  Board 

10,664 

0.09% 

Telecommunications  &  Information 

6,343 

0.05% 

Taxi  Commission 

5,606 

0.05% 

War  Memorial  Commission 

24,712 

0.20% 

Total 

$12,395,938 

100.00% 
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City  Services  Auditor  Resources 

The  Controller's  Office  has  six  divisions  and  an  overall  budget  of  $30.4  million  with  the  City  Services 
Auditor  making  up  approximately  41  percent  of  that  amount.  CSA  has  approximately  49  full  time 
equivalents  including  auditors,  performance  analysts,  project  managers  and  operations  staff. 

Controller's  Office  Budget  by  Division,  Fiscal  Year  2009-10 

Economic 
Analysis,  $.03M, 
1% 


Public  Finance, 
$0.5M,  2% 


CSA  Full-Time  Equivalent  Employees  By  Unit,  Fiscal  Year  2009-10 
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IV.  Major  Planned  Audits  and  Projects  for  Fiscal  Year  2009-10 

Listed  below  are  all  audits  and  projects  budgeted  at  $50,000  or  more  planned  by  CSA  for  fiscal  year 
2009-10.  In  addition  to  the  major  efforts  listed  here,  CSA  has  many  smaller  audits  and  projects  as 
part  of  its  ongoing  programs. 


Audit/Project  Name  Budget 
Airport 

Airline  Landing  Fee  Monitoring,  Concession  and  Lease  Management 

Functions  Performance  -  Audit  ' 

Airport  Concession  &  Airline  Compliance  -  Audit(s)  58,000 


Children,  Youth  &  Their  Families 

Department  of  Children,  Youth  and  Their  Families  Grant  Expenditures  ,-n  nnn 

and  Grant  Process  -  Audit  5U,uuu 

Performance  Outcomes  for  Grant  Programs  -  Project  70,000 

Program  Officer  Time  Study  and  Risk-based  Monitoring  Analysis-  Project  70,000 


ClTYWIDE  AND  MULTI-DEPARTMENT 

Audit  Consulting  -  As  Needed  63,700 

Audit  Follow-ups  FY  201 0  70,000 

Better  Streets  Implementation  -  Project  99,600 

Budget  Projects  -  As  Needed  100,000 

Coordinated  Case  Management  System  Implementation  -  Project  310,000 

Citywide  Cash  Disbursements  -  Audit  1 00,000 

Citywide  Nonprofit  Monitoring  and  Capacity  Building  -  Project  336,100 

Citywide  Performance  Measurement  System  -  Project  240,000 

Comcast  Franchise  -  Audit  52, 1 25 

Contracting  Reform  -  Project  200,000 

Information  Technology  -  Audit(s)  60,000 

Labor  Negotiation  s  Support  Citywide  -  Project  50,000 

Labor  Negotiations  Support  MOU  Review  -  Project  50,000 
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Audit/Project  Name 


Budget 


ClTYWIDE  AND  MULTI-DEPARTMENT  (CONTINUED) 

Long-term  Care  Community  Alternatives  -  Fiscal  and  Gap  Analysis  -  Project 

Mayor's  Community  Based  Organizations  Task  Force  Report  and 
Implementation  -  Project 

Office  Depot  Billing  Review  -  Audit 
Payroll  Audit(s)  for  FY09-10 
Performance  Barometer  -  Project 

Public  Safety  Processing:  Efficiency  and  Effectiveness  -  Project 
Whistleblower  -  Program 


$50,000 
50,000 
60,200 
60,000 
50,000 
100,000 
300,000 


Controller 

Senate  Bill  (SB)  90  Claiming  Project  -  Project 


72,000 


Department  of  Public  Health 

Behavioral  Health/Primary  Care  Integration  -  Project 

Department  of  Public  Health  Consultative  Planning  Process  -  Project 

Federally  Qualified  Health  Centers  (FQHC)  Reimbursement  Rate  Analysis  - 
Project 

Laguna  Honda  Model  of  Care  and  Organizational  Effectiveness  -  Project 

Laguna  Honda  Transition  Assistance  -  Project 

Medicare  Reimbursement  -  Audit 

New  Patient  Appointment/Triage  Line  -  Project 

Private  Provider  Network  Evaluation/Technical  Assistance  -  Project 

San  Francisco  General  Hospital  Information  Technology  Transition  Planning 
-  Project 

San  Francisco  General  Hospital  Throughput  -  Project 

Short  Doyle  Revenue  Enhancement-  Project 

Substance  Abuse  Treatment  Services  Evaluation  -  Project 


$230,000 
209,000 
50,000 
311,500 
197,500 
50,000 
80,000 
50,000 
250,000 
250,000 
150,000 
897,000 
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Audit/Project  Name  Budget 
Department  of  Public  Works 

Indirect  Cost  Allocation  Plan  -  Project  $60,000 

Street  and  Sidewalk  Perception  Study-  Project  75,000 

Police 

Organizational  Change  Implementation  and  Ingleside  Pilot  -  Project  375,474 

Human  Services  Agency 

Analysis  of  Financial  Stability  of  Care  Not  Cash  -  Project  50,000 

Cost-Performance  Linkage  Pilot  -  Project  50,000 

Document  Imaging  Cost-Benefit  Analysis  -  Project  100,000 

Evaluation  of  Interpreter  Contract  Services  -  Project  50,000 

Social  Security  Income  (SSI)  Advocacy  Models  Comparison  -  Project  80,000 

Staffing  Analysis:  Accounting  Positions  -  Project  60,000 

Municipal  Transportation  Agency 

Garage  Audit  -  Moscone  and  Performing  Arts  -  Audits  50,000 

Garage  Management  and  Oversight  Performance  -  Audit  50,000 

Garage  Audit  -  Pacific  Park  Management  -  Audit  65,325 

Proposition  E  -  Transportation  Quality  Review  -  Project  50,000 

Succession  Planning  for  Critical  Areas  of  MTA  -  Project  50,000 

Transit  Effectiveness  Project  Implementation-  Project  500,000 

Port 

Port  Concessions  -  Audit(s)  67,000 

Lease  Administration-Accounting  &  Accounts  Receivable  -  Audit  50,000 
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Office  of  the  Controller,  City  Services  Auditor 
Annual  Work  Plan,  Fiscal  Year  2009-10 


Audit/Project  Name  Budget 
Public  Utilities  Commission 

Cost-Performance  Linkage  Pilot  -  Project  $50,000 

Selected  Non-Percentage  Leases  -  Audit(s)  60,000 

Water  System  Improvement  Program  (WSIP)  Local  Construction  -  Audit(s)  185,000 

Recreation  and  Park  Department 

Park  Bond  Programmatic  -  Audit  60,000 

Park  Management  Plans  -  Project  50,000 

Park  Standards  Program  -  Project  1 50,000 
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